The present invention relates, in general, to image processing systems and, more specifically, to a method and apparatus for fixating a camera on a target point using image alignment.
During video surveillance or filming of a scene from a moving vehicle, such as an aircraft or car, a common problem is how to keep the camera centered on a stationary target in the scene. Such centering is desirable, because it results in the video looking stable and, therefore, easy to watch; video that does not stay locked on a point contains significant motions due to the moving platform. The motion makes the video look xe2x80x9cjitteryxe2x80x9d and may even cause most of the desired scene to move out of the image sporadically. This problem is especially acute in situations where the field of view is narrow and the camera platform is subject to sudden changes in orientation, such as airborne surveillance.
The task of keeping a camera centered on a point is commonly referred to as xe2x80x9cfixatingxe2x80x9d a point. Traditional approaches to fixating a point have primarily relied on mechanical stabilization to hold the camera steady. Typically, these approaches use gyroscopes to sense rotation of the camera platform, and motors to compensate for that rotation. However, the cost of such systems increases rapidly depending on the number of rotational axes, and the desired precision of the stabilization.
Recent approaches have added an electronic image matching device to supplement mechanical stabilization. The device operates on a small image region (typically of size 32xc3x9732 pixels) around the image center, and uses correlation matching to estimate the translational image motion of this region from one video frame to the next. With correlation matching, the amount of xe2x80x9cimage slipxe2x80x9d may be measured. Image slip is the distance through which the center of the current image moves from the desired fixation point. This image slip may be converted to an angular correction and applied to a motorized pan/tilt camera mount in order to re-center the camera on the desired fixation point. In this manner, fixation may be achieved by a combination of mechanical stabilization and closed-loop pan/tilt control that uses correlation matching to measure the image slip. An implicit assumption of this correlation matching approach is that the patch motion in the image can be represented using a rigid  less than x,y greater than  translation. If the patch motion differs from a pure translation, or if the patch contains insufficient texture, the motion estimates produced using this technique will contain errors. As these errors accumulate over time, the patch and the fixated point will drift off the target.
As is well known, there are video techniques for indicating moving objects from a stationary platform that operate by simply subtracting from the amplitude of each pixel of a current video image frame, the amplitude of that pixel in a stored preceding video image frame. This results in the substantial removal of those pixels that define stationary objects. Digital image processing techniques which may be employed in this frame difference approach are disclosed in U.S. Pat. No. 4,692,806, which issued on Sep. 8, 1987, and to Anderson et al., and is incorporated herein by reference.
Also incorporated herein by reference is an article by Burt et al. xe2x80x9cObject tracking with a moving camera: an Application of Dynamic Motion Analysis,xe2x80x9d IEEE Workshop on Visual Motion, Irvine Calif., March 1989. Burt et al. teaches that if the camera is moving, it is often possible to compensate for resulting image motion by electronically shifting and rotating successive image frames to achieve alignment prior to computing a frame difference. Electronic alignment is based on a simple image warp (e.g., based on an affine transform) that is effective when scene motion contains relatively little parallax, such as when the camera is rotating, but not translating, or when objects within the region of the camera""s field of view occur in a relatively narrow range of depths.
Further incorporated herein by reference is U.S. Pat. No. 5,629,988 issued to Burt and Hanna on May 13, 1997. This patent discloses a method for removing unwanted camera motion from processed and displayed video. It matches image frames between successive camera images, or between images and a stored reference image to determine camera-induced motion.
None of these disclosures, however, address the problem of fixation in which a selected target point is to be maintained centered in the camera""s field of view. The image alignment technique of the present invention is directed to overcoming the problem of fixation, especially in the presence of motor latencies.
To meet this and other needs, and in view of its purposes, the present invention provides a method and apparatus for fixating a camera on a target point using image alignment. In one embodiment, the method includes the following steps:
a) receiving a plurality of images representative of a scene; b) selecting, within the plurality of images, a first display reference image containing the target point at a first coordinate location; c) estimating, in a current image within the plurality of images, a transformation that maps the current image to the first display reference image; d) estimating a second coordinate location of the target point in the current image using the transformation; e) computing an image slip between the target point in the current image and the target point in the first display reference image; f) converting the image slip into an angular correction for fixating the camera; g) modifying the angular correction by using closed-loop control with the motorized mount; and h) warping the current image using the transformation to align the current image with the first display reference image to obtain a stable view.
It is understood that the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the invention.